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Summary 

The U.S. Army Crew Station Research and Development 
Branch (CSRDB) of the Aircraft Simulation Division, 
AVSCOM, was tasked by the Light Helicopter Program 
Manager (LH-PM) to provide training to Army personnel 
in advanced aircraft simulation technology. The purpose 
of this training was to prepare different groups of pilots to 
support and evaluate two contractor simulation efforts 
during the Demonstration/Validation (DEM/VAL) phase 
of the LH program. The personnel in the CSRDB devel- 
oped mission-oriented training programs to accomplish 
the objectives, conducted the programs, and provided 
guidance to Army personnel and support personnel 
throughout the DEM/VAL phase. The conduct of these 
training programs was partially supported by Contract 
NAS-2- 12849. 

Background 

The Request for Proposal for the DEM/VAL portion of 
the LH development program contained requirements for 
engineering, part-task, and full mission simulation. The 
full mission simulation description specified that the 
Government assess the contractors’ simulations with 
regard to the adequacy of the operational realism of the 
simulators. To assist the contractor teams, the government 
was asked to assign a group of pilots to each of the 
contractors during the entire DEM/VAL program for 
employment as Subject Matter Experts (SMEs) in the 
development of the cockpits to be used for simulation at 
each contractor site. These Support Pilots were AH-64A 
and Army Helicopter Improvement Program (AHIP) 
(OH-58D) pilots who were trained at the Crew Station 
Research and Development Facility (CSRDF) in two 
groups during March and April of 1989. A separate group 
of pilots was required to perform the operational assess- 
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ment of the contractor simulators in tactical scenarios. 
These Assessment Pilots were also AH-64A and AHIP 
pilots who were trained at CSRDF in three distinct phases, 
each with a different objective. 

A group of Engineering Test Pilots was trained with the 
Assessment Pilots at the CSRDF. Their role was to assess 
the flight handling qualities of the contractors’ simulators 
compared with ADS-33 standards. 

Operational personnel responsible for developing, imple- 
menting, and orchestrating the scenarios required for the 
operational assessment were provided support, training, 
and practice in their tasks at the CSRDF. These personnel 
included the Test Director for the Operational Assessment 
and members of the Operational Assessment Team. Tech- 
nical personnel responsible for developing, orchestrating, 
and testing the technical scenarios required for the Tech- 
nical Assessment were provided support, academic and 
technical training, and practice in their tasks at the 
CSRDF. These personnel comprised an Engineering Test 
Pilot, a Research Analyst functioning as a copilot and 
Systems Operator, and the other members of the 
Technical Assessment Team. 

Strategy 

Both the support and assessment groups consisted of 
operational pilots with little or no exposure to the envi- 
ronment of glass cockpits, advanced Mission Equipment 
Packages (MEPs), and full mission simulators. For this 
reason, the pilot training program was primarily directed 
toward providing familiarization with the advanced tech- 
nologies currently proposed or available for such a 
cockpit. “Technology Familiarization” was the primary 
goal of both the Support Pilot Training program and the 
First phase of the Assessment Pilot Training program. 

The training strategy resulted in an approach that would 
familiarize the pilots with advanced cockpit technologies 
in general. To avoid a strong tendency on the part of the 
pilots to judge the contractor’s simulator relative to the 
one on which they were trained, the cockpit controls and 
displays of the CSRDF were not emphasized, even though 


the CSRDF simulator was the only simulator the pilots 
flew during the training program. 

During technology familiarization, the pilots were pre- 
sented general concepts and high level information about 
various advanced technologies in classroom lecture for- 
mat. A concerted effort was made, however, to present a 
variety of examples of how these technologies have been 
implemented. The pilots were reminded that the examples 
presented may or may not represent good implementations 
of those technologies. This approach was intended to 
diminish the tendency to judge the MEPs of the contrac- 
tors’ simulators relative to any other simulator MEP, and 
to express judgement of a MEP on its own merit, 

Throughout Assessment Pilot training, data were gathered 
on the pilots’ performance. These data were used in two 
ways: (1) as a basis for assigning equivalent groups of 
pilots to each contractor; and (2) to detect extremes in 
performance by any of the pilots. 

Support Pilot Training Program 

Training of the Support Pilots (Subject Matter Experts) 
associated with the LH DEM/VAL program was accom- 
plished in the Spring of 1989. The goal of the four-day 
course was to familiarize the operational pilots with the 
advanced technologies proposed or available for the LH 
glass cockpit environment. Lectures were presented in 
Simulation Basics and Technology Familiarization. The 
pilots were exposed to advanced technology system 
components available in the CSRDF. 

The twelve Support Pilots participated in the training 
program in two groups of six pilots each. An additional 
group of six pilots attended the training program; four 
were trained with the First group of Support Pilots, and 
two were trained with the second group. These additional 
pilots were involved with the LH DEM/VAL in other 
roles and attended the lectures which were presented, but 
their involvement with other training activities was 
allowed on a non-interference basis (i.e., simulator flight 
time was arranged only when it had no impact on the 
Support Pilots’ scheduled time). Table 1 summarizes the 
four-day training program. 

Because of the short training time, only a brief exposure 
to the Crew Station Facility could be accommodated. The 
feedback received from this program indicated the pilots 
learned a great deal about glass cockpits, but would have 
preferred more hands-on time with all events, such as 
Computer-Based Training, the Smart Command Recog- 
nizer, and EXPERT87. These topics will be described in 
more detail in the Assessment Pilot Training section 
which follows. 


Table 1. Support pilot training program summary 


Topic 

Total time, 
hr 

Introductory materials, administrative 

5.50 

briefings, discussions, facility description 
and tours 

Introduction to simulation lectures 

4.00 

Technology familiarization lectures 

9.00 

Helmet fitting 

0.50 

EXPERT87 

2.00 

Mission planner demonstration 

1,50 

Computer-based training 

2.00 

Crew station flight time 

2,00 

Team station flight time 

2.00 

Experimenter-operator console observation 

0.50 

Mission briefing/discussion 

1.50 


Assessment Pilot Training Program 

The Assessment Pilot Training was accomplished in three 
distinct phases, totalling five weeks of training time for 
each of twenty pilots. The goals of the overall program 
were: (1) to provide a general introduction to simulation 
(Simulation Basics); (2) to familiarize pilots with 
advanced glass cockpit technologies (Technology Famil- 
iarization); (3) to provide pilots with hands-on experience 
flying advanced cockpits; and (4) to provide experience in 
simulation protocol. Lectures were presented in Simula- 
tion Basics and Technology Familiarization, addressing 
similar content to that which was taught to the Support 
Pilots. Phase I training was an expanded version of the 
Support Pilot Training Program. Some additional activi- 
ties were added to the Assessment Pilot program, e.g., 
each pilot had a custom-fit helmet made for flying at the 
CSRDF, and more attention was paid to the mission 
scenarios to be flown at the contractors’ sites. 

Phase I 

Two groups of ten pilots attended the two-week course for 
Phase I Assessment Pilot training. One group was trained 
in November, and the second group was trained in 
December, 1989. The curriculum included classroom lec- 
tures in Simulation Basics and Technology Familiariza- 
tion, hands-on experience using Computer-Based 
Trainers, demonstration and practice using the Helicopter 
Operations Planner (HOP) for mission planning, demon- 
stration and use of the Smart Command Recognizer used 
in the CSRDF, acquisition of mission effectiveness 
judgements using EXPERT87, Team Station operations. 
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and CSRDF training for both front and back seats. See 
table 2 for a summary of the Phase I training. 

The classroom lectures were provided to present in-depth 
information about Simulation Basics and Technology 
Familiarization. These lecture periods were distributed 
throughout the two-week course, with tours and hands-on 
experiences interspersed to maintain interest and attention 
levels. The lectures were accompanied by slide displays 
and videotape presentations. The major topics addressed 
under the Simulation Basics category were: Visual Sys- 
tems, Motion and Fixed-Base Simulators, Simulator 
Sickness, Crew Coordination, and Performance Mea- 
surements. Data relevant to simulator side effects (e.g., 
simulator sickness) were collected throughout the training 
program. 


Table 2. Assessment pilot Phase I training summary 


Topic 

Total time, 
hr 

Introductory materials, administrative 

7.75 

briefings, discussions, facility description 


and tours 


Introduction to simulation lectures 

3.00 

Technology familiarization lectures 

6.25 

Introduction to SWAT and card sort 

2.00 

Helmet pouring and adjustment 

1.75 

Voice system enrollment 

1.00 

Template polishing 

1.50 

Speech command practice 

0.75 

EXPERT87 

1.00 

Mission planner practice 

1.00 

Computer-based training 

4.50 

Crew station flight time 

7.50 

Team station flight time 

7.50 

Experimenter-operator console observation 

1.50 

Scenario discussions 

2.50 


The major topics addressed under the Technology Famil- 
iarization category were: Flight Controls, Helmet- 
Mounted Displays, Multi-Function Displays, Communica- 
tions and Data Link Capabilities, Aided Target Recogni- 
tion, Weapons Management, Digital Maps and other 
Navigational Aids, Sensor Systems (FLIR), Speech Input 
and Output, and Mission Planning. As with the Simulation 
Basics materials, training aids were used and demonstra- 
tions provided whenever they were available and 
appropriate. 

Computer-based trainers- The Computer-Based 
Trainers (CBTs) were used to teach system level tasks 


associated with operating the two principle displays in the 
CSRDF. Four training stations were available so that 
pilots could go through exercises in small groups. Each 
training station included two multi-function displays with 
touch-sensitive screens, which were similar in presenta- 
tion and operation to the displays in the CSRDF. Pilots 
were required to perform a structured sequence of 
exercises which began with tutorial demonstrations and 
progressed to presentation of interactive problem sets. The 
problem sets provided brief questions concerning the 
operation of different CSRDF subsystems. Pilots 
responded by touching the appropriate keys on either or 
both of the displays. When an answer was completed, the 
CBT presented feedback to the pilot in terms of a score 
describing how well they answered each question and 
timing feedback. The problem sets were intended to test 
the pilots' knowledge of the subsystems he would be 
required to operate in the CSRDF. Pilots were required to 
progress through the entire lesson plan once, and were 
permitted to repeat any tutorials or problem sets. 

Helicopter operations planner- The Helicopter Opera- 
tions Planner (HOP) was used to demonstrate mission 
planning capabilities. The HOP plans ingress and egress 
routes based on threat positions, density, and the altitude 
at which different segments of the mission must be flown 
(e.g., nap of the earth, contour). The pilots were presented 
background information concerning integrated mission 
planners, and were given a demonstration to illustrate how 
the HOP could be used to plan missions. 

Smart command recognizer- The pilots were shown the 
capabilities and limitations of current speech recognition 
and synthesis technology. Speech input and output design 
features were illustrated. The Smart Command Recog- 
nizer (SCR) was employed to enroll speech templates for 
the pilots and to train them to use a set of speech com- 
mands in the Crew Station Facility. Speaker-dependent 
templates were developed which included a 108- word 
vocabulary. Commands were available to control commu- 
nications, the digital map, and to request flight and 
weapon stores information. The SCR features flexible 
command wording, feedback of spoken commands, and 
methods for correcting recognizer errors. Through the use 
of the SCR, the pilots also became familiar with the 
synthesized speech messages that they might hear in the 
Crew Station. 

EXPERT87- EXPERT87 is a PC-based software pack- 
age originally developed to be a decision-aiding tool. 
Starting with a high-level construct such as Mission 
Effectiveness, EXPERT87 facilitates the specification of 
factors that would be possible contributing factors to the 
construct (e.g., for Mission Effectiveness these factors 
might be performance in each of the following areas: 
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Navigation/Pilotage, Communications, Targeting/Threat 
Management, etc.). Once these have been defined, the 
software determines the importance placed on each of 
these factors by an individual pilot in several interactive 
sessions. The DEM/VAL pilots were presented with 
information about the background and intent for using 
EXPERTS? in a classroom, and then allowed to spend 
about an hour using the package. 

Team station operation- Pilots were given instructions 
concerning the operation of the Blue and Red Team 
Stations which played roles as wingmen or enemy aircraft 
in the practice missions flown by the pilots during this 
training phase. During CSRDF mission flights, the pilots 
each gained experience operating the Team Stations. 

Crew station operation- Each pilot was given an oppor- 
tunity to operate the CSRDF simulator from both the front 
and back seats. While in the front seat, the pilots were 
given instructions for flying the simulator. When the 
pilots were in the back seat they were provided with 
instructions and guidance in operating the cockpit controls 
and displays, weapons systems, and target acquisition 
sensors. Pilots flew a short mission, allowing them to 
obtain an overview of capabilities and functions. 

Phase II 

Four groups of five pilots attended the two-week course 
for Phase II Assessment Pilot training. The first group of 
pilots was trained in January and the last group of pilots 
was trained in April, 1990. The curriculum included class- 
room lectures in Tactical Standard Operating Procedures, 
Threat Disposition, pre-mission briefings; hands-on expe- 
rience using the Computer-Based Trainers, and extensive 
CSRDF simulator and Team Station flight training. In 
addition, the pilots went through an EXPERT87 session 
and participated in other related research projects. See 
table 3 for a summary of the Phase II training. 

The classroom lectures were provided to present informa- 
tion concerning tactical requirements for different types of 
missions. Except for the first day of training, the pilots 
received one lecture period each day. The first training 
day was spent covering administrative items and provid- 
ing review sessions for the material covered in Phase I 
training. The pilots completed all of the CBT exercises 
and contributed to EXPERT87 on the first day. Lecture 
topics included Armed Reconnaissance, Light Attack, 
Threat Logic, and Air-To-Air Combat. 

Computer-based trainers- The Computer-Based Train- 
ers were used to provide refresher training for the CSRDF 
simulator controls and displays. No demonstrations were 
presented; only interactive problem sets were used. The 
pilots were given a pre-test to establish a baseline of their 


Table 3. Assessment pilot Phase II training summary 


Topic 

Total time, 
hr 

Lectures, TAC SOP, threat briefings 

10 

Mission preparation briefings 

7 

Computer-based training 

2.5 

Review and discussions 

5 

Front seat crew station flight time 

9 

Back seat crew station flight time 

9 

Team station flight time 

9 

EOC observation 

5 

ATR study participation 

6 

HMD study participation 

2 


retained knowledge. Following that, two problem sets 
posed questions to the pilots and provided step-by-step 
solutions to the problems. In contrast, two other problem 
sets posed questions without providing answers, and 
finally, the pre-test was administered again (as a post-test) 
to judge the effectiveness of the training. This process 
required approximately 3 hours to complete. 

Crew station training- A Crew Station Refresher course 
was presented before the pilots spent any time in the 
cockpit, and the CBT training was also conducted before 
CSRDF simulator flight training. The first day in the 
cockpit was scheduled to re-acquaint the pilots with the 
systems. The time was unstructured and was designed to 
relax the pilots in the simulator. Each pilot flew once 
every day in both the front and back seat of the CSRDF 
simulator. 

The next four days of training concentrated on different 
cockpit subsystems in a building block format. Pilotage 
and Navigation was stressed first, followed by Communi- 
cations and Battle Resource Management. Targeting and 
Aircraft Survivability Equipment were emphasized next, 
and finally Weapons Management was practiced. Each 
day’s exercises included reviews of tasks from the previ- 
ous days. The training tasks were specifically tailored for 
each seat and for the Team Station player. For example, 
the Pilotage and Navigation exercises required the Battle 
Captain (in the back seat) to enter waypoints and plan a 
route to perform a simple flight objective. The Pilot (in 
the front seat) was given flight instructions by an Instruc- 
tor Pilot (TP) to learn how to fly the simulator. When the 
Pilot had accomplished the required maneuvers, the Battle 
Captain guided him to the planned route and the two of 
them worked on cockpit/crew coordination to accomplish 
the objective. This training plan continued with the 
addition of tasks related to the other cockpit subsystems. 
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Finally, a short practice mission was flown, prior to flying 
a Reconnaissance Mission and a Light Attack Mission. 
The pilot’s tactical performance was rated following each 
of these two missions. 

Study participation- Two research projects not related to 
the LH DEM/VAL program were conducted during the 
Phase II training program and pilots were required to 
contribute part of their time to these projects. One study 
was concerned with assessing user acceptance of an Aided 
Target Recognition by using an emulation of a proposed 
system. The other study sought pilot input for determining 
the criteria important to display of aircraft system 
symbology on a helmet-mounted display. 

Phase HI 

Phase III training was conducted just prior to the actual 
Operational Assessment at the contractor sites. The pilots 
were assigned to a contractor team before participating in 
Phase III training (see Pilot Assignment Process section). 
Two groups of ten pilots each attended the four-day 
course. The first group was trained in June, and departed 
the CSRDF to attend a two-week training program at the 
First Team’s simulation site in Connecticut. The second 
group was trained in July before going to Arizona to 
attend a two-week training program at the Super Team’s 
simulation site. For both groups, the contractor’s Opera- 
tional Assessment immediately followed the contractor 
training program. 

The purpose of the Phase III training at the CSRDF was to 
train assigned crews in the crew positions that they would 
fly during the actual Operational Assessment. The pilots 
comprising each crew flew in the same position for each 
mission. (Note that during Phases I and II, all pilots flew 
in both seats.) Each two-man crew consisted of an AH-64 
pilot who flew in the front seat, and an OH-58D pilot who 
flew in the back seat. Two additional pilots operated as 
wingmen for the missions. The pilots flew four tactical 
vignettes which closely resembled those that would be 
used for the operational assessment. Two reconnaissance 
and two attack missions were flown over the four days; 
one mission was flown by each of the four crews each 
day. The two-man crews worked as teams to plan each 
mission using the Helicopter Operations Planner, and they 
briefed the wingmen regarding the role they were to play 
for each mission. Following each mission, a tactical 
debriefing was conducted. 

The curriculum for Phase III included classroom briefings 
in Subjective Workload Assessment Technique use, 
Aircraft Survivability Equipment operations, and Threat 
situation. These topics were addressed because of their 
importance in the conduct of the Operational Assessment. 


Other topics included: demonstration and use of the HOP 
for pre-flight route planning; CSRDF or Team Station 
flight time; and participation in on-going research activi- 
ties. See table 4 for a summary of the Phase III training. 


Table 4, Assessment pilot Phase III training 


summary 


Topic 

Total time, 
hr 

Administrative briefings, lectures 

6.25 

Crew station missions 

8.0 

Mission planning 

2.0 


At the end of Phase III training, the Assessment Pilots 
were prepared to continue at the contractors’ sites, where 
specific training for the contractors’ simulators was 
provided by both contractors. 

Pilot Assignment Process 

Immediately after Phase II training was completed, the 
twenty DEM/VAL Assessment Pilots were divided into 
two groups which would be subsequently assigned to one 
of the two contractor teams competing on the LH pro- 
gram. The CSRDB was tasked to establish the criteria for 
assigning the pilots to two comparable groups. The objec- 
tive was to achieve a balanced representation of 
experience, background, and other relevant variables. 

Constraints on the assignment process were as follows: 

• Four pilots from Aviation Development Test Activity 
(ADTA) were assigned as two crews, one crew to each 
contractor group. 

• Two pilots representing the TRADOC System Manager 
(TSM) were a priori designated to be team station 
operators, one assigned to each group. 

Because these constraints accounted for six of the twenty 
pilots, the decision rules pertained only to the assignment 
of the remaining fourteen pilots into two groups of seven 
pilots. Each group consisted of three two-man crews (two 
primary crews and one back-up) and one team station 
operator. Final determination of the back-up crew and the 
two primary crews within each group was made at the 
contractor site following the contractor’s training 
program. 

Approach- The general approach to pilot assignment was 
as follows: 
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1 . Assign pilots to two groups on the basis of hard data 
(i.e., aircraft type and tactical flight hours). 

2. Assign pilots to crews on the basis of quantified sub- 
jective ratings made by Army Subject Matter Experts 
(SMEs). 

3. Test the equity of the outcome by subjecting it to 
critical review by independent SMEs. 

4. Allow the TRADOC Systems Manager (TSM), LH to 
resolve any issues raised by the critical review, and to be 
responsible for assigning the final groups to the specific 
contractor teams. 

Assignment criteria- Relevant criteria for pilot 
assignment were determined by representatives from 
Aeroflightdynamics Directorate (AFDD), Test Command 
(TEXCOM), Director of Combat Development (DCD), 
Army Materiel Systems Analysis Activity (AMSAA), and 
Operational Test and Evaluation Agency (OTEA) in a 
meeting at NASA Ames Research center on January 22, 
1990. The variables were selected on the basis of being 
quantitative (or quantifiable) measures relevant to the 
projected role of the LH DEM/V AL assessment pilots. 

The two variables used for initial division into two 
balanced groups were: aircraft type (AH-64 vs OH-58D), 
and tactical flight experience. Data pertaining to these 
variables were readily obtained from existing pilot 
profiles supplemented by a brief interview session with 
the pilots regarding unit assignments and duty positions. 

Other variables which contributed to the assignment 
criteria were: mission success, tactical aptitude, situation 
awareness, flying ability, decision making, Mission 
Equipment Package (MEP) usage, and attitude. Four 
Army SMEs from TEXCOM, DCD, AMSAA, and TSM 
provided judgments of relative weights (importance) of 
these variables, separately, for the Pilot and Battle 
Commander crew positions. The mean ratings were 
calculated to establish the variable weights. 

Three Army SMEs rated the Assessment Pilots on the 
seven listed variables during four full-mission simulations 
in the CSRDF during Phase IT Assessment Pilot training. 
Each pilot flew twice in the front seat and twice in the 
back seat, and was rated on his performance. The pilot 
rating data were computed using a decision matrix to 
weigh and sum the SME ratings of the assessment pilots. 
This decision matrix produced a single composite score 
for each pilot. These scores were used to determine pilot 
pairings. 

Pilot assignment process- The fourteen FORSCOM 
pilots were separated by aircraft type (AH-64 and 
OH-58D) into two groups of seven. These two groups 
were sorted by total number of tactical flight hours, from 


high to low. From these two groups, one pilot was 
excluded based on the percentage difference between his 
tactical flight hours and the tactical flight hours of the 
pilots who had the most similar number of tactical flight 
hours (i.e., the pilots who had just more or just less total 
tactical flight hours according to the hierarchical sort). 

Each of the remaining three pairs of pilots in each aircraft 
type were split to determine assignment to Contractor A 
and B. The assignment to A or B was done by tactical 
flight hours in the following order, beginning with the 
highest number: 

AH-64 Pilots = AB BA BA 

OH-58D Pilots = BA AB AB 

Crew pairings- Crew pairs were created through the use 
of the ratings obtained in Phase II CSRDF training. For 
Group A and B independently, the composite rating 
scores of the three pilots from each aircraft type were rank 
ordered. The middle scoring pilots were paired; the high- 
est scoring AH-64 pilot was paired with the lowest scor- 
ing OH-58D pilot; and the lowest scoring AH-64 pilot 
was paired with the highest scoring OH-58D pilot. 

Assignment process overrides- The Army SMEs, led by 
the TEXCOM representative, reserved the right to adjust 
crew assignments and pairings to avoid apparent 
inequities or logistical problems. Two changes were made 
to the group assignment process by the Army SMEs. One 
of the OH-58D pilots who was assigned as a console 
operator was switched with another OH-58D pilot 
assigned as one of the three crews for one team. The 
SMEs determined that the pilot originally assigned to the 
console could be equated (on the basis of tactical flight 
hours) with another pilot because the absolute difference 
in their flight hours (as opposed to percentage difference 
in tactical flight hours) was less than the absolute differ- 
ence between the two pilots based on the original assign- 
ment process. In addition, the pilot who would have been 
selected through the original assignment process had a 
serious problem with simulator sickness during his train- 
ing at the CSRDF, and required medication to avoid 
becoming sick. The medication slowed his reaction times 
and affected his decision-making performance in the 
Crew Station. 

The original assignment process placed two Department 
of Evaluation and Standardization (DES) Pilots on the 
same team. The second change was made to avoid this. 
The OH-58D DES Pilot assigned to Team A was switched 
with his counterpart (in terms of tactical flight hours) in 
Team B. 

Critical review process- After the assignment process 
was completed, the outcome was subjected to a critical 
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review. Three Army Aviation SMEs were identified by 
the TSM, LH to form a “Red Team.” They were provided 
with a list and description of the pilots assigned to the two 
groups. The groups were not affiliated with either 
contractor and individual pilots were not identified by 
name. 

The Red Team determined that both groups were evenly 
matched along biographical parameters such as age and 
rank. Professional qualifications were also matched. The 
discriminators for matching the two groups identified by 
the Red Team were: total rotorcraft hours, tactical flight 
hours, simulator hours, and the subjective analysis of 
individual pilot performance by the SMEs during Phase II 
training. The most critical discriminators according to the 
Red Team were tactical flight hours and simulator hours. 

The Red Team analysis of the two groups showed no 
significant difference in tactical flight hours. One group 
had 50% more simulator time than the other group. This 
difference was attributable to one pilot who had 900 hours 
in fixed-wing, commercial simulators. The Red Team did 
not judge this experience to be pertinent to the Simulation 
Assessment. On the basis of rotary wing military simula- 
tor time, there was no difference between the two groups. 

A small, and statistically insignificant, difference was 
shown between the two groups in the SME subjective 
ratings of pilot performance during Phase II training. 
These data were not viewed as decisive by the Red Team. 

The Red Team concluded that the two groups of pilots 
provided for optimum matching based on the discrimina- 
tors utilized. They recommended that each group should 
be randomly assigned to each contractor, and that these 
groups of pilots should be used to conduct the Simulation 
Assessment (i.e., they recommended no changes to the 
groups on the basis of their review). 


Assignment of groups to contractors- After the two 
groups were identified, the TSM, LH was responsible for 
determining which group went to which team. This 
decision could have been made by either a random 
process, or based on logistical constraints. 

Conclusions 

Each of the two groups of pilots were trained at the 
contractor sites in preparation for the Operational 
Assessment. The training they received was adequate to 
prepare them for their tasks as Operational Assessment 
pilots. 

The Operational Team Members (the Test Director for the 
Operational Assessment and his support personnel) 
received sufficient training at the CSRDF to understand 
and orchestrate the mission scenarios which were to be 
evaluated. They obtained a good understanding about the 
operational use of a simulator facility. 

The Technical Team Members also received adequate 
training in the technical aspects of simulation to prepare 
them for their mission objectives. The Technical Team 
dry runs conducted at the CSRDF were extremely valu- 
able in preparing the team members in the use of time 
management, methodology, data collection and evaluation 
methods. 

The DEM/VAL training program proceeded smoothly and 
accomplished the stated objectives: operational pilots 
were trained to support the DEM/VAL phase of the LH 
program, mission-oriented programs were developed, and 
guidance was provided to Army personnel to support the 
DEM/VAL program. The CSRDF staff benefitted by 
learning more about Army tactics and doctrine, and by an 
ongoing association with Army personnel to accomplish 
similar objectives from different perspectives. 
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